Method for determining optimal viewpoints for 3D face modeling and face recognition

ABSTRACT

A method determines an optimal set of viewpoints to acquire a 3D shape of a face. A view-sphere is tessellated with a plurality of viewpoint cells. The face is at an approximate center of the view-sphere. Selected viewpoint cells are discarded. The remaining viewpoint cells are clustered to a predetermined number of viewpoint cells according to a silhouette difference metric. The predetermined number of viewpoint cells are searched for a set of optimal viewpoint cells to construct a 3D model of the face.

RELATED APPLICATION

This is a Continuation-in-Part Application of U.S. patent applicationSer. No. 10/636,355 “Reconstructing Heads from 3D Models and 2DSilhouettes,” filed on Aug. 7, 2003 now U.S. Pat. No. 7,212,664, by Leeet al.

FIELD OF THE INVENTION

This invention relates generally to image processing, and moreparticularly to modeling and recognizing faces according to 3D modelsand 2D images.

BACKGROUND OF THE INVENTION

In computer graphics, it is still a fundamental problem to syntheticallyconstruct realistic human heads, particularly the face portion.Hereinafter, when referring to ‘head’ or ‘face’, the invention is mostinterested in that portion of the head extending from chin-to-brow, andear-to-ear. Most prior art methods require either extensive manual laborby skilled artists, expensive active 3D scanners, Lee et al., “RealisticModeling for Facial Animations,” Proceedings of SIGGRAPH 95, pages55-62, August, 1995, or the availability of high quality of textureimages as a substitute for exact face geometry, see Guenter et al.,“Making Faces,” Proceedings of SIGGRAPH 98, pages 55-66, July 1998, Leeet al., “Fast Head Modeling for Animation,” Image and Vision Computing,Vol. 18, No. 4, pages 355-364, March 2000, Tarini et al., “TexturingFaces,” Proceedings Graphics Interface 2002, pages 89-98, May 2002.

To acquire 3D models for human faces by active sensing requires costlyscanning devices. Therefore, a number of techniques have been developedto recover the 3D shape of faces from 2D images or ‘projections’. Someof those methods are based on a direct approach, which obtains 3Dlocation of reference points on the face using dense 2D correspondencesof the images, P. Fua, “Regularized bundle-adjustment to model headsfrom image sequences without calibration data,” International Journal ofComputer Vision, 38(2) pp. 153-171, 2000, F. Pighin, J. Hecker, D.Lischinski, R. Szeliski, and D. Salesin, “Synthesizing realistic facialexpressions from photographs,” Proceedings of SIGGRAPH 98, 1998. and Y.Shan, Z. Liu, and Z. Zhang, “Model-based bundle adjustment withapplication to face modeling,” Proceedings of ICCV 01, pp. 644-651, July2001.

Other methods parameterize 3D face models, and search for optimalparameters that best describe the 2D input images, V. Blanz and T.Vetter, “Face recognition based on fitting a 3D morphable model,” PAMI,25(9), 2003, J. Lee, B. Moghaddam, H. Pfister, and R. Machiraju,“Silhouette-based 3D face shape recovery,” Proc. of Graphics Interface,pp. 21-30, 2003, and B. Moghaddam, J. Lee, H. Pfister, and R. Machiraju.“Model-based 3D face capture using shape-from-silhouettes,” Proc. ofAdvanced Modeling of Faces & Gestures, 2003.

In either case, the number of viewpoints and 2D input images is animportant parameter for high quality 3D model reconstruction.Intuitively, the more input images that are taken from differentviewpoints, the higher the quality of the 3D model and subsequentreconstructions. But, that increases processing time and the cost ofequipment.

However, if an optimal set of viewpoints can be determined, then itbecomes possible to use a smaller number of cameras and their resulting2D images provide better 3D modeling accuracy.

Up to now, a systemic method for determining the optimal number ofviewpoints and, thus, input images, for the purpose of constructing a 3Dmodel of a face has not been available. It would also be advantageous toselect automatically specific images out of a sequence of images in avideo, the selected images corresponding to optimal viewpoints toimprove face recognition.

It is known that different objects have different prototype or aspectviewpoints, C. M. Cyr and B. B. Kimia, “Object recognition using shapesimilarity-based aspect graph,” Proc. of ICCV, pp. 254-261, 2001.

It is desired to determine a canonical set of optimal viewpoints for aspecific class of objects with notably high intra-class similarity,specifically the human face.

When dealing just with illumination, it is possible to determineempirically an optimal configuration of nine point sources of lightwhich span a generic subspace of faces under variable illumination, K.Lee, J. Ho, and D. Kriegman, “Nine points of light: Acquiring subspacesfor face recognition under variable lighting,” Proc. of CVPR, pp,519-526, 2001.

It is desired to solve a related problem for subject pose, orequivalently camera viewpoint. That is, it is desired to determine anoptimal set of K viewpoints corresponding to a spatial configuration ofK cameras that best describe a 3D human face by way of projections fromthe viewpoints, i.e., shape silhouettes in 2D images.

SUMMARY OF THE INVENTION

A fundamental problem in multi-view 3D face modeling is to determine aset of optimal viewpoints or ‘poses’ required for accurate 3D shapeestimation of a ‘generic’ face. Up to now, there is no analyticalsolution to this problem. Instead partial solutions require a nearexhaustive combinatorial search.

Based on a 3D modeling method, the invention uses a contour-basedsilhouette matching method and extends the method by aggressive pruningof a view-sphere with viewpoint clustering, and various other imagingconstraints. A multi-view optimization search is performed using bothmodel-based (eigenheads) and data-driven (visual hull) methods, yieldingcomparable sets of optimal viewpoints.

The set of optimal viewpoints can be used for acquiring the 3D shape offaces, and provide useful empirical guidelines for the design of 3D facerecognition systems.

Because no analytical formulation is possible, the invention uses anempirical approach. The view-sphere about the object is sampled(tessellated) to generate a finite set of viewpoint configurations. Eachviewpoint is evaluated according to a resulting ensemble error on arepresentative dataset of individual faces. The ensemble error is interms of an average reconstruction error.

Due to the large number of potential viewpoints, the view-sphere ispruned aggressively by discarding a predetermined set of irrelevant orimpractical viewpoints, which can depend on the application. Theinvention can use aspect viewpoints for general 3D object recognition.An aspect viewpoint is a projection of a silhouette of an object from aviewpoint, which represents a range of similar nearby viewpoints in aspace of the uniformly sampled view-sphere. However, the invention isnot limited to using aspect views as such, because the method can workwith any set of pre-determined views that are deemed to be salient.

A size of the viewpoint space is reduced for a class of objects, e.g.,faces. After uniformly sampling the view-sphere and applying high-levelmodel-specific constraints, such as facial symmetry and imaginggeometry, the method generates viewpoint clusters by merging nearbyviewpoint cells using a silhouette difference metric, or inversely, asimilarity metric, and selects prototypical “centroids” of each clusteras aspect viewpoints. A search of the reduced number the combinatorialsubsets of these aspect viewpoints for a given number of distinct views(cameras) then constitutes the set of optimal viewpoints for modelingthe shape of the object.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is an image of an original face in a database;

FIG. 1B is a resampled image of the face of FIG. 1A;

FIG. 1C is an image of a 3D model of a head obtained by scanning;

FIG. 1D is an image obtained by merging the resampled image of FIG. 1Bwith the model of FIG. 1C;

FIG. 2A is a tessellated view-sphere;

FIG. 2B is the tessellated view-sphere with discarded viewpoints;

FIG. 3 is a view-sphere with clustered viewpoints;

FIG. 4 are silhouettes obtained from ten aspect viewpoints;

FIG. 5 is a block diagram of a method according to the invention; and

FIG. 6 is a diagram of a view-sphere according to the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Multi-View 3D Face Modeling

Our present invention provides a method for determining an optimal setof viewpoints required to construct an accurate 3D model of a human faceacquired from 2D images taken from the set of viewpoints. Our generalmethod for constructing the 3D model is described in U.S. patentapplication Ser. No. 10/636,355 “Reconstructing Heads from 3D Models and2D Silhouettes,” filed on Aug. 7, 2003, by Lee et al, incorporate hereinby reference.

As shown in FIG. 6, our method uses an arrangement of many cameras 600,e.g., eleven, placed on ‘view-sphere” 200 around a person's head 210.The placement of the cameras determines the size of the portion of thehead that is modeled. In actual practice, the view-sphere is constructedas a geodesic dome with the cameras fitted to the structural members ofthe dome. The person sits inside the dome on a chair while images areacquired of the person's face. The face 210 is at an approximate centerof the view-sphere 200.

As described therein, our 3D modeling method treats the recovery ofshape, i.e., geometry, independently of texture, i.e., appearance.Hence, our method is robust with respect to variations in lighting andtexture. Our method is distinguishable from the ‘morphable’ models, V.Blanz and T. Vetter, “Face recognition based on fitting a 3D morphablemodel,” PAMI, 25(9), 2003, in the following ways.

Shape is recovered directly, and not jointly with an estimation oftexture. Shape is obtained from occluding contours or silhouettes, andnot from the texture. Estimation of object texture is not required.However, texture can easily be obtained from the object after the shapeis recovered using standard techniques. Our model-fitting uses binarysilhouettes, and not image intensity errors. Our method does not requireactual images per se. The method can use any other means offoreground/background segmentation of depth layer information. Forexample, silhouettes can be obtained with range sensors.

Furthermore, our silhouette-matching optimization is simpler, has fewerfree parameters and is considerably faster, by a factor of approximatelyten.

In our prior modeling method, an optimal arrangement of the cameras 600was initially found by trial and error, and using our “intuition” as towhich viewpoints are more informative for acquiring shape.

Here, we continue our invention.

It is now our goal to remove the guess-work from the viewpoint selectionprocess, and determine an optimal geometry or view configuration for agiven number of K cameras. For model building, our method used scans ofmale and female adult faces of various races and ages. The scans can beused to produce meshes. The number of points in each face mesh variesapproximately from 50,000 to 100,000.

All scanned faces in the database are resampled to obtain point-to-pointcorrespondences. Second, the resampled faces are aligned to a referenceface to remove any variations in pose variation, or any misalignmentduring the scan. Third, we perform principal component analysis (PCA) onthe database of aligned 3D faces to obtain eigenvectors of our shapemodel and their associated eigenvalues, i.e., variances of theirimplicit Gaussian distribution. This decomposition can be used toreconstruct new or existing faces through a linear combination of“eigenhead” basis functions, see generally, J. J. Atick, P. A. Griffin,and N. Redlich, “Statistical approach to shape from shading facesurfaces from single 2D images,” Neural Computation, 8(6) pp. 1321-1340,1996.

An inspection of the PCA eigenvalue spectrum and the resulting shapereconstructions indicates that the first sixty eigenheads are sufficientfor capturing most of the salient facial features of faces in thedatabase. Therefore, the corresponding shape parameters a_(i) are ouroptimization parameters.

An arbitrary face model M(a) produces a polygon mesh given a parametervector a={a₁, a₂, . . . , , a_(n)}. Input silhouette images are S^(k)_(input), for k=1, . . . , K. A similarity transformation T aligns areference model face to a real 3D face. A silhouette image S^(k)_(model)(a) is a rendered by projecting T(M(a)) onto an image planeusing pose information in the k^(th) silhouette image. The parametervector a is estimated by minimizing a total penalty

$\begin{matrix}{{{E(a)} = {\sum\limits_{k = 1}^{K}\;{f( {S_{input}^{k},{s_{model}^{k}(a)}} )}}},} & (1)\end{matrix}$where the cost function ƒ measures a difference between two binarysilhouettes. For the cost function ƒ in Equation (1), a simpledifference metric between two binary silhouettes is the number of ‘on’pixels when a pixel-wise exclusive-or (XOR) operation is applied.

To prioritize matching the correct pixels on occluding contours and topromote uniqueness so that the cost function ƒ has a global minimum, weimpose a higher penalty for any mismatch near boundary pixels of theinput silhouette. Specifically,

$\begin{matrix}\begin{matrix}{{{f( {S_{input}^{,k},\;{S_{model}^{k}(a)}} )} = {\sum\limits_{i}^{H}\;{\sum\limits_{j}^{N}\;{c( {i,j} )}}}},} \\{{c( {i,j} )} = \{ \begin{matrix}0 & {{{if}\mspace{14mu}{S_{input}^{k}( {i,j} )}{S_{model}^{k}(a)}( {i,j} )},} \\{d( {i,j} )}^{- 2} & {{otherwise},}\end{matrix} } \\{{{d( {i,j} )} = {{{D( S^{k} )}( {I,j} )} + {{D( \overset{\sim}{S} )}( {i,j} )}}},}\end{matrix} & (2)\end{matrix}$where D(S) is a Euclidean distance transform of binary image S, andimage {tilde over (S)} is an inverse image of image S. Note that thevariable d represents a distance map from the silhouette contour. Thevariance can be determined after a preprocessing step. We call this costfunction a boundary-weighted XOR. The cost function provides a simpleand effective alternative to precise contour matching.

Consequently, there is no need for time consuming processing ofcorrespondences with edge-linking, curve-fitting and distancecomputations between contours. Furthermore, the boundary-weighted XORoperations can be performed in hardware. Given the inherent complexityand nonlinearity of the cost function and no analytic gradients, we usea probabilistic downhill simplex method to minimize Equation (1).

Determining Optimal Viewpoints for 3D Face Modeling

We now continue our invention by describing a method to determine a setof optimal viewpoints for 3D face modeling using an arbitrary number Kof cameras or ‘viewpoints’, for example, five or less. We describe howto ‘prune’ the space of all possible viewpoints, obtained by uniformtessellation of the view-sphere, based on clustering adjacent viewpointsusing a silhouette difference or similarity metric obtained from shapeprojections. The selected set of aspect viewpoints is then examinedusing both our model-based method and a data-driven visual hull method.

Silhouette Generation

The silhouettes of a resampled face in our database are quite differentfrom the silhouettes obtained from images of actual subjects. This isdue to missing portions of the head and upper torso. To simulatesilhouette images of actual subjects with our database, we use a fullyscanned 3D head as our prototype head/torso.

FIG. 1A is an image of an original face in the database, FIG. 1B is animage of a resampled face, FIG. 1C is an image of a laser scanned full‘prototype’ head, and FIG. 1D is an image of a rendered face obtained bymerging the resampled face with the scanned full head.

The merging is done by aligning the facial region of the prototype headto the resampled face by smooth deformations, and “stitching” the headand face together to synthesize a “virtual” test subject, complete withfull head and shoulders. Therefore, we can generate complete silhouetteimages with the same exact face shapes as in the database, whilemaintaining the proper geometry of the subjects.

This pre-processing step is only used in lieu of having complete headscans and can be omitted when complete subject scans are available. Theprocess of “stitching” our 3D face models to one common head shape onlyhighlights the critical facial area as the key “region-of-interest” inthe subsequent analysis and optimization. By doing so, we areeffectively indicating that non-critical areas, such as the back of thehead, etc., are not important or salient for accurate reconstruction ofthe face area. The search methodology, however, can remain the sameregardless of which areas are highlighted, i.e., marked as salient ornot.

View-Sphere Tessellation

As shown in FIG. 2A and FIG. 5, we tessellate 510 the view-sphere 200uniformly with triangles using a subdivision of a dodecahedron aroundthe subject 210. This procedure yields one-hundred-and-twenty triangles201, which we call viewpoint cells. The vertices 202 of each triangle201 are on a surface of the view-sphere 200.

As shown in FIG. 2B and FIG. 5, we discard 520 selected viewpoint cells.The discarded viewpoint cells include cells in the rear-hemisphere ofthe view-sphere, with respect to the camera, because the face isoccluded from those viewpoints. We further discard the viewpoint cellswhich are above and below a predetermined elevation, because thecorresponding viewpoints are unlikely or impractical physical locationsfor a camera. In our method, we restrict the elevation of viewpoints to±45° from a central horizontal plane.

Furthermore, it is often very difficult to acquire accurate facialcontour from oblique viewpoints, due to the occlusion and resultingconfusion hair and shoulders. Finally, we discard an entire half of theremaining viewpoints due to an approximate bilateral symmetry of faces.This leaves forty-four viewpoints shown in FIG. 2B.

The remaining viewpoints still result in too many combinations orsubsets of viewpoints. For example, to find eleven optimal viewpoints byan exhaustive search there are approximately 7×10⁹ viewpointcombinations to evaluate. This is quite intractable. Therefore, we needto further reduce the search space even further.

Clustering Viewpoints

Our observation is that 2D silhouette images of two neighboringviewpoints are often substantially similar. Therefore, we measure asilhouette difference for two neighboring viewpoints and cluster 530 thetwo corresponding viewpoint cells when the silhouette difference is lessthan a predetermined threshold.

A location of a group (cluster) of viewpoint cells can then berepresented by the centroid of the cluster of viewpoint cells. Moreimportantly, here we consider only the silhouette differences near thecritical facial areas, e.g., nose, eyes, ears, chin, and mouth, becauseface shape recovery is not affected by the silhouette differences inother irrelevant areas, such as the shoulders.

For clustering, we first build a lookup table (D) that stores a partialor face-restricted XOR silhouette distance between every pair ofviewpoints in the search space. Initially, every viewpoint is considereda cluster, and the aspect viewpoint of the cluster is the viewpointitself.

We define the silhouette difference between two clusters by thesilhouette distance between their aspect viewpoints. That information ispre-computed and stored in the look-up table D. We find the two neighborclusters that have a minimum silhouette difference among all the otherneighbor clusters and merge these clusters. After merging two clusters,we determine a new aspect viewpoint for the new merged cluster. The newaspect viewpoint is the viewpoint that has the minimum value for themaximum silhouette difference compared to all the other viewpoints inthe same cluster. We repeat this process until a predetermined number ofclusters remain.

FIG. 3 shows ten clusters 1-10 and approximate corresponding aspectviewpoints 300 obtained using the clustering step 530. Note that theresulting aspect viewpoints are not necessarily geometric centroids ofclusters, but rather, viewpoints with a minimum silhouette difference toother members of the cluster.

To circumvent any subject-dependency and to generalize this clustering,all the entries in our lookup table D are generated by averaging thepair-wise silhouette difference distances for fifty differentsynthesized heads in our database.

Table A gives the coordinates of the aspect viewpoints 1-10 whereinazimuths of {90°, 0°, +90°} correspond to {left, front, right}directions in a head-centered reference frame.

TABLE A View # Azimuth ^(O) Elevation ^(O) 1 3.4 40.4 2 7.6 −15.5 3 28.2−17.0 4 31.4 18.9 5 40.0 0.9 6 48.3 −16.5 7 52.2 16.8 8 55.1 39.4 9 63.1−30.2 10 85.9 17.7

FIG. 4 shows the corresponding silhouettes 401-410 obtained form the tenaspect viewpoints, along with the model silhouette, and the criticalfacial area used for error evaluation. All reconstruction errors areconfined to the critical facial area. Extraneous inputs from hair andshoulders are ignored. We discard view 1 in FIG. 3. Because of thedownward angle, the corresponding face silhouette 401 is partiallyhidden and confounded by the torso. View 2 is also discarded becausefrontal veiwpoints offer very little occluding contour as constraintsfor shape recovery although frontal viewpoints are preferred foracquiring facial texture.

Determining Optimal Viewpoints

Given the remaining eight aspect viewpoints 3-10, we search 540exhaustively for the optimal subset of K≦8 viewpoints, which for eachcase K, yield a closest 3D shape reconstruction with respect to theoriginal face, using the K silhouettes for the shape recovery process.The default reconstruction method is our model-based (eigenhead) 3D faceshape recovery method, as described in the related U.S. patentapplication Ser. No. 10/636,355.

For comparison, we also tested a purely data-driven method using avisual hull construction method. It should be noted that visual hulls bythemselves are not suited for accurate reconstructions, even withhundreds of viewpoints. Our goal is to show that a greedy search basedon a data-driven method selects a similar set of optimal viewpoints.

For the set of optimal viewpoints to be relevant for general purposeface modeling and recognition, the viewpoints should apply for genericfaces of all kinds, e.g., gender, ethnicity, age. Therefore, optimalityshould be independent of the subject. To this end, we used arepresentative subset of twenty-five individuals from our database andbased our optimal viewpoint selection on the con-figurations thatminimized the total or averaged error for all subjects.

When we recover a 3D shape from silhouette images, we require a metricthat measures the error between the ground truth and the reconstructed3D geometry. Because our focus is on the facial area of the recoveredshape, we need a metric that measures the difference in the criticalfacial area of the recovered shape and the original face. The basicapproach for this error measurement is as follows.

The first step is to find a dense point set on the facial area of therecovered face geometry. With an eigenhead shape model, we find thefacial points for our model via a mesh parameterization.

However, it is not trivial to find the same facial points on a visualhull. We use a ray casting method to find the facial points on thevisual hull. Because we have images of the original 3D heads, which weuse to generate the input silhouette images from facial points on theoriginal head, we cast rays toward the visual hull and get correspondingsamples on a surface of the visual hull.

After we obtain the facial points, we use the same ray casting scheme toget the corresponding samples on the surface of a ground truth mesh. Wemeasure the L2 distances of the facial points on the recovered face andthe corresponding points on the ground truth and use the L2 distances asthe 3D error metric for the facial area.

Model-Based Reconstructions

We performed the exhaustive search 540 on the eight remaining aspectviewpoints in FIG. 4 to find the set of optimal subset of viewpoints forK={1, 2, 3, 4, 5} cameras. Therefore, the total number of possiblereconstructions is 5450. To remove the data dependency inherent in asingle individual's reconstruction error, we use the averagereconstruction error of twenty-five subjects that are selected randomlyfrom the database.

The results are presented in Table B, which shows the set of optimalviewpoints for K={1, 2, 3, 4, 5}, and the corresponding minimum averagereconstruction errors, refer to Table A for exact coordinates of theaspect viewpoints.

TABLE B Best Best Standard Error Error K Viewpoints Error Deviation MeanStd Dev 1 4 40.7 12.4 45.0 3.3 2 3, 10 31.9 8.6 37.6 4.3 3 3, 5, 10 28.26.2 31.7 2.9 4 3, 4, 9, 10 26.8 6.2 31.7 2.9 5 3, 4, 7, 8, 10 26.6 7.130.2 2.2

The standard deviation of the individual errors for all twenty-fivesubjects under the best configuration is also presented. The averageerror means and average error standard deviations are based on theaverage reconstruction errors across all viewpoints. Both tend todecrease with increasing K as expected because more viewpoints providemore constraints.

Visual Hull Reconstructions

Using the same search strategy as described above for the 3D model basedmethod, we now evaluate the visual hull constructions obtained from thegiven subset of silhouette images and compare the results to the groundtruth.

Table C shows the optimal viewpoints for K={2, 3, 4, 5} and thecorresponding error values. The visual hull from a single silhouette(K=1) has no finite volume, and is omitted.

TABLE C Best Best Standard Error Error K Viewpoints Error Deviation MeanStd Dev 2 3, 10 418.7 26.1 847.7 400.4 3 3, 9, 10 406.0 24.7 643.5 246.94 3, 8, 9, 10 399.9 25.8 541.0 163.3 5 3, 4, 8, 9, 10 398.3 25.5 481.2108.9

Note that a visual hull reconstruction, especially one from a smallnumber of images, is not a very accurate representation. Unlike ourmodel-based results, here the reconstruction quality is much moreview-dependent than subject-dependent. However the view dependencydecreases significantly as the number of viewpoints (K) increases, seethe error standard deviations. For both methods, viewpoints 3 and 10seem to be the most informative.

EFFECT OF THE INVENTION

The method according to the invention determines a set of optimalviewpoints for 3D face modeling, in particular, methods that recovershape from silhouettes. The invention provides useful guidelines fordesigning 3D face recognition systems and are in agreement with existingpractice and intuition. For example, the most salient viewpoint 3corresponds very closely with the established biometric standards ‘¾view’, which is used for many identification photographs, and viewpoint10 corresponds to the profile view used in ‘mugshot’ photographs. Ourresults indicate that reconstructions do not improve significantlybeyond four to five viewpoints, see the best errors listed in Tables Band C.

It is possible to incorporate additional physical and operationalconstraints into our method. For example, although a direct frontalviewpoint is not very salient for shape, it is the preferred view forcapturing texture, hence this view is used by nearly all 2D facerecognition systems. This viewpoint can be pre-selected before thesearch.

In video-based face acquisition, motion of the subject and posevariation provide multiple virtual viewpoints, even though the camera isfixed. Therefore, our method can be applied to a sequence of images in asurveillance video to select automatically the optimal poses, i.e.,video frames that are best for face recognition.

Although the invention has been described by way of examples ofpreferred embodiments, it is to be understood that various otheradaptations and modifications may be made within the spirit and scope ofthe invention. Therefore, it is the object of the appended claims tocover all such variations and modifications as come within the truespirit and scope of the invention.

1. A method for determining an optimal set of viewpoints to construct a3D shape of a face from 2D images acquired at the optimal set ofviewpoints, comprising: tessellating a view-sphere with a plurality ofviewpoint cells, where the face is located at an approximate center ofthe view-sphere; discarding selected viewpoint cells; clusteringremaining viewpoint cells to a predetermined number of viewpoint cellsaccording to a silhouette difference metric determined from the 2Dimages; and searching the predetermined number of viewpoint cells forthe optimal set of viewpoints.
 2. The method of claim 1, in which thetessellation uses triangles.
 3. The method of claim 1, in which thetessellation is a uniform subdivision of a dodecahedron.
 4. The methodof claim 1, in which the selected viewpoint cells include viewpointcells in a rear-hemisphere of the view-sphere.
 5. The method of claim 1,in which the selected viewpoint cells include viewpoint cells above andbelow a predetermined elevation.
 6. The method of claim 5, in which thepredetermined elevation is ±45° from a central horizontal plane of theview-sphere.
 7. The method of claim 1, in which the selected viewpointcells include viewpoint cells of half of the face due to an approximatebilateral symmetry of two halves of the face.
 8. The method of claim 1,in which the silhouette difference metric is measured for each pair ofviewpoint cells.
 9. The method of claim 1, in which the predeterminednumber of viewpoint cells is ten.
 10. The method of claim 1, in which alocation of the viewpoint cells is determined by a centroid of thecluster of viewpoint cells.
 11. The method of claim 1, in which only thesilhouette differences near critical facial areas are considered. 12.The method of claim 11, in which the critical facial areas include anose, eyes, ears, a chin, and a mouth.
 13. The method of claim 1, inwhich the silhouette distances are stored in a precomputed look-uptable.
 14. The method of claim 1, in which the searching is exhaustive.15. The method of claim 1, in which the set of optimal viewpoint cellsis applied to a sequence of images in a video to select automaticallyoptimal poses for face modeling.
 16. The method of claim 1, in which theset of optimal viewpoint cells is applied to a sequence of images in avideo to select automatically optimal poses for face recognition. 17.The method of claim 1, in which the set of optimal viewpoint cells isused to construct the 3D model of the face from 2D images acquired atthe optimal set of viewpoints.
 18. The method of claim 1, in which thesearching is a combinatorial search of all possible subsets of theoptimal set of viewpoint cells.
 19. The method of claim 1, in whichoptimality is defined by a minimum reconstruction error between an inputface and the 3D model of the face.
 20. The method of claim 19, in whichthe minimum reconstruction error is based on a combination of shape andtexture.
 21. The method of claim 19, in which the minimum reconstructionerror is based on an average of a plurality of faces.